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ABSTRACT 


This  thesis  analyzes  problems  in  current  Naval  aircraft  as  reported 
by  fleet  aviators  attending  the  U.  S.  Naval  Aviation  Safety  School.  A 
method  is  developed  which  facilitates  the  collection  and  processing  of 
the  reported  information.  A collective  sample  of  286  incidents  is  strati- 
fied into  a design  discrepancy  outline  which  illuminates  31  specific  prob- 
lem areas.  Various  recommendations  are  made  concerning  concept 
expansion  to  a fleet-wide  level. 
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I.  INTRODUCTION 


The  systematic  study  of  documented  aircraft  problems  experienced 
by  operational  personnel  in  the  field  receives  little  allocation  of  the 
human  factors  research  being  done  in  the  Navy  today.  This  seems  sur- 
prising when  one  pauses  to  reflect  that  military  aerial  operations  are 
composed  of  complex  man-machine  systems  functioning  in  extremely 
competitive  situations.  The  environment  which  pits  a competitor  versus 
an  opponent  (whether  an  actual  physical  enemy  or  simply  nature)  is 
likely  to  contain  vicissitudes  as  unsettling  as  imaginable.  Yet  most 
expect  maximum  mission  performance  from  the  system  at  all  times  and 
place  demands  upon  it  accordingly.  A necessary  condition  for  such 
optimum  performance  is  that  man,  the  system's  most  variable  and  least 
predictable  component,  must  be  allowed  to  apply  his  diverse  skills  to 
the  situation  at  hand  without  distraction.  Yet,  in  the  actual  evolution 
of  a mission,  system  elements  supporting  man  can  operate  below  opti- 
mum performance  at  any  given  time.  It  is  this  suboptimum  perform- 
ance which  can  distract  an  aviator's  attention  from  a critical  task  and 
lead  to  error  and  mission  performance  degradation.  Usually  these 
errors  are  attributed  to  the  human.  However  this  paper  adopts  an 
alternative  approach,  namely  considering  inadequate  aircraft  design 
as  a major  contributing  factor  to  suboptimum  performance.  It  is  well 
within  the  realm  of  the  Human  Factors  discipline  to  study  these  errors, 
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determine  causes,  develop  procedures /methods  for  alleviation,  and 
finally  to  communicate  the  results  to  the  design  engineer. 

Thus  there  is  a need  for  a human  factors  method  which  can  sys- 
tematically collect,  classify,  and  analyze  data  specifically  chosen  to 
aid  the  designer  in  both  identifying  and  solving  man- machine  interface 
problems.  Meister  and  Sullivan  (1967)  concluded  that  current  human 
factors  information  supplied  in  the  design  stage  is  entirely  "too  verbal, 
too  general  and  non-applicable.  " In  fact,  Meister  and  Farr  (1966) 
reported  in  an  earlier  study  of  the  perception  designers  have  of  such 
data: 

"Human  Factors  problems  - which  are  not  usually  verbalized 
as  such  - are  settled  largely  by  the  designer's  placing  him- 
self in  the  role  of  the  equipment  operator.  Designers  have 
great  difficulty  in  anticipating  operational  problems  that  may 
result  from  design  parameters  and  are  unable  to  apply  evalua- 
tive criteria  to  completed  designs.  " 

More  recently  Wherry  (1975)  pointed  out  that  current  techniques, 

".  . . rely  heavily  on  the  expertise  of  the  human  engineer 
to  provide  'good1  estimates  for  the  data  required  by  the 
various  techniques  and  methods  ...  if  the  information 
input  into  the  technique  is  unreliable  and  invalid  then  the 
output  results  will  also  be  unreliable  and  invalid.  " 

Thus  it  is  clear  that  a data  pipeline  is  sought  which  could  provide  the 

design  engineer  with  more  objective  information  concerning  operational 

problems  than  he  could  generate  subjectively  himself. 

A question  that  might  be  asked  in  a search  for  such  data  is,  "What 

parameters  significantly  influence  operator  performance  in  the  field?  " 

The  present  thesis  represents  one  approach  to  this  question,  using 
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responses  elicited  from  the  users  for  obtaining  evaluative  data  on 
operational  systems.  It  is  felt  that  any  form  of  evaluation  in  system 
development  should  include  human  considerations  as  well  as  engineering. 

In  the  area  of  Naval  aircraft  cockpit  design  this  implies  that  Naval 
Aviators  and  Naval  Flight  Officers  (NFO'S)  should  be  canvassed  for 
reports  of  design  discrepancies.  It  is  recognized  that  reports  of  design 
I problems  submitted  by  the  fleet  personnel  have  been  a basis  for  limited 

modification  in  the  past.  Yet  too  often  the  human  engineer  has  been 
faced  with  single  reports  with  which  he  must  decide  if  a discrepancy 
is  an  isolated  problem  or,  in  fact,  truly  fleet-wide.  The  collection  and 
classification  of  a large  number  of  fleet  generated  design  discrepancies 
is  a necessary  step  in  providing  a sound,  documented  basis  for  modify- 

1 

ing  existing  equipment  and  designing  new  systems. 

The  type  of  information  volunteered  by  fleet  operators  is  highly 
subjective  in  nature.  This  led  to  a search  of  existing  techniques  for  a 

procedure  to  classify  and  analyze  such  data.  This  thesis  will  demon- 

I . | 

strate  that  the  Problem  Incident  Technique  can  successfully  organize 

I 1 

this  new  information  into  categories  that  can  aid  the  Navy  in  four  areas: 

1.  To  improve  the  quality  of  decisions  in  developing  new  Naval 
aircraft. 

2.  To  aid  the  design  engineer  during  periods  of  retrofit  of 
existing  Naval  aircraft. 

3.  To  provide  better  integration  of  hardware  and  personnel 
(the  man-machine  system)  in  Naval  aircraft. 

4.  To  correct  design  deficiencies  early  in  development  programs 
of  new  aircraft. 

U 
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II.  BACKGROUND 


Since  World  War  II  military  systems  have  undergone  dramatic 
periods  of  retrofit  and  reorientation  during  their  operational  lives. 
These  changes  have  occurred  in  response  to  such  variables  as  changes 
in  the  threat,  technology  alterations  involving  new  or  upgraded  equip- 
ment, or  of  the  deployment  of  systems  in  new  operational  environments. 
There  is  little  doubt  that  operators  in  the  field  would  have  benefited 
during  these  periods  from  a comprehensive  analysis  of  their  perceived 
problems . 

Such  an  analysis  across  various  Naval  aircraft  types  could  also 
aid  the  design  engineer  faced  with  evolving  a weapon  systems  develop- 
ment program  for  new  aviation  platforms.  As  an  example  consider  the 
approach  to  such  a program  which  has  been  proposed  by  Wherry  (1975) 
and  is  presented  in  Figure  1. 

This  program  extends  from  the  premise  that  the  only  defensible 
rationale  behind  the  development  of  future  airborne  weapon  systems 
is  to  counter  enemy  threats  as  perceived  for  some  given  time  frame. 
Thus  the  first  phase  consists  of  defining  the  operational  requirements 
needed  to  counter  the  expected  threat  parameters.  The  next  phase, 
mission  scenario  requirements,  discusses  how  the  Navy  might  per- 
ceive the  engagement  with  enemy  forces  unfolding  with  time.  Third, 
system  capability  requirements  are  described  based  on  the  various 
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FIGURE  1 

AIR  WEAPON  SYSTEMS  DEVELOPMENT  PROGRAM 
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scenarios.  Finally  the  subsystem  requirements  may  be  defined  as 
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"an  aggregation  of  components  to  accomplish  various  functions  which 
taken  together  provide  portions  of  the  needed  system  capabilities" 

(Wherry,  1975).  This  may  also  be  viewed  as  the  first  of  the  design 
stages  since  it  conceptualized  how  the  system  might  be  organized.  It 
is  at  this  level  that  the  value  of  supplying  fleet  generated  data  surfaces. 
Here  it  can  be  blended  with  (see  Figure  1.  ) human  capability  and  engineer- 
ing state-of-the-art  inputs  to  provide  a real  world  sieve  through  which 
design  concepts  can  be  screened. 

No  claim  is  made  that  all  man-machine  problems  can  be  caught  by 
such  an  approach.  But  fleet  responses  can  provide  a foundation  for 
identifying  and  diagnosing  many  otherwise  ignored  problems  which 
affect  mission  success  (as  determined  by  the  humans  charged  with 

4 

eliciting  performance).  It  is  recognized  that  most  critical  and  urgent 
problems  do  not  exist  unheeded  for  long  in  the  normal  weapon  system 
design  process.  However,  this  technique  can  potentially  identify  and 
document  them  much  earlier  - prior  to  the  prototype  stage  and  thus 
prior  to  the  need  for  an  expensive  Engineering  Change  Proposal.  Also 
it  could  highlight  at  the  outset  factors  which  are  ordinarily  not  document- 
ed in  operational  field  reporting  and  which  when  later  identified  as 
troublesome  downstream  in  the  design  process  are  not  deemed  worth 
the  expense  to  change. 
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The  practical  goal,  the  payoff,  in  developing  fleet  generated  data 
for  identifying  critical  problem  areas  involves  the  reduction  of  system 
performance  errors.  Error  may  be  defined  as  simply  a mistake,  a 
deviation  from  correct  procedure,  which  results  in  lost  time,  damaged 
equipment,  personal  injury,  or  any  form  of  mission  degradation. 

The  Human  Factors  attitude  towards  error  tends  toward  the  premise 
that  you  cannot  divorce  the  man  from  the  system  designed  for  him.  Man 
is  a system  component  - and  both  the  strongest  and  weakest  element  in 
the  aviation  system  at  that.  He  is  the  strongest  because  he  is  the  most 
flexible  and  adaptable  in  a given  situation.  He  is  the  weakest  because 
his  performance  is  unreliable;  he  cannot  perform  the  same  task  again 
and  again  in  an  identical  manner.  Nonetheless,  man  must  be  relied 
upon  for  effective  performance  if  the  system  is  to  accomplish  its  tasks. 
To  this  end  the  goal  of  Human  Factors  is  to  optimize  the  design  of 
equipment  so  that  the  user's  efficiency  will  be  at  its  peak. 

Although  this  thesis  employs  a method  which  concentrates  on  errors 
in  equipment  design  rather  than  human  error  itself,  it  is  necessary  to 
realize  that  a causal  relationship  exists  between  the  two.  Ricketson, 
Kennamore,  and  Callen  (1975)  provide  a functional  definition  of  pilot 
(human)  error  and  its  causal  elements  (see  Figure  2). 


VEHICLE/ 


FUNCTIONAL  DEFINITION  OF  PILOT  ERROR 
FIGURE  2 


Items  one  through  eight  identify  basic  environmental  elements  which 
influence  the  aviation  system  and  are  potentially  error  producing.  Item 
two  is  highlighted  in  Figure  2 since  it  represents  the  focus  of  this  paper, 
namely  vehicle/ equipment  design.  When  any  of  these  elements  exceeds 
its  normal  tolerance  level,  the  pilot  must  correct  for  this  aberration 
while  continuing  to  perform  his  present  ongoing  tasks.  This  can  result 
in  an  overload  condition  for  the  aviator  which,  if  it  occurs  at  a critical 
moment,  can  induce  error.  The  outcome  will  be  various  degrees  of 
performance  degradation. 

A similar  viewpoint  was  expressed  by  Meister  (1971).  He  deter- 
mined that  three  general  causes  of  error  could  induce  an  overload 
condition.  First,  error  could  arise  because  of  an  operator's  psycho- 
logical or  physiological  state  - fatigue,  lack  of  capability,  lack  of 
motivation.  These  were  called  "operator-induced"  errors.  Second, 
lack  of  proper  maintenance  programs,  supervisory  practices,  facilities, 
or  any  other  deficiency  in  total  system  planning  could  result  in  "system- 
induced"  errors.  Finally  a state  conducive  to  human  errors  could  be 
induced  upon  the  aviator  by  design  problems  in  individual  equipment; 
these  were  termed  "design-induced  errors. 

It  is  the  third  category  of  error,  henceforth  called  design  discrep- 
ancies, which  is  highlighted  and  analyzed  by  the  Problem  Incident 
Technique.  These  are  simply  problems  with  equipment  design  charac- 
teristics which  create  special  difficulty  for  the  operator  and  which 
substantially  increase  the  potential  for  error. 
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IV.  HISTORICAL  PERSPECTIVE 
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The  employment  of  techniques  utilizing  field  type  data  to  analyze 
system  performance  problems  is  not  a new  development  in  military 
aviation.  Fitts  and  Jones  (1947a)  were  early  investigators  of  such 
methods  and  analyzed  460  incidents  of  pilot  error  involving  the  opera- 
tion of  aircraft  controls.  Later  Fitts  and  Jones,  (1947b)  collected  and 
analyzed  270  pilot  error  experiences  in  reading  and  interpreting  aircraft 
instruments.  Their  findings  concluded  that  the  design  of  aircraft  equip- 
ment must  take  into  account  the  capabilities  and  limitations  of  the  human 
operator. 

Later  applications  of  the  technique  involved  a flight  safety  orientation. 
Korchin  and  Patterson  (1949)  collected  497  accounts  of  "close  calls" 
which  were  reported  by  pilots  for  an  evaluation  of  flight  safety  media. 
Vasilas  et  al  (1953)  defined  these  "close  calls"  as  "near  accidents"  and 
devised  a standard  report  form  for  collection  of  Air  Force  personnel 
data.  Significantly,  they  concluded  that  a group  orientation  approach 
(defined  in  section  V.b.  ) was  a highly  effective  method  of  data  collection 
for  such  a study.  Ricketson  et  al  (1973)  surveyed  accident  investigation 
reports  for  U.  S.  Army  aviation  over  a fifteen  year  period.  They  devel- 
oped an  experimental  human  error  reporting  form  in  the  course  of  their 
study  and  concluded  that  the  resultant  data  held  promise  for  providing 
a clearer  identification  of  mishap- causing  system  elements. 
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Rabideau  and  Ritchie  (1968)  employed  open-ended  questioning  of 
Air  Force  personnel  in  interview  situations  for  a study  of  human  engineer- 
ing problems  incurred  during  tactical  air  operations  in  Southeast  Asia. 

A classification  of  problem  commonalities  and  causative  factors  resulted 
in  more  than  two  dozen  problem  areas.  Atkins  (1969)  attempted  to  collect 
data  inputs  for  developing  new  and  revised  criteria  of  Air  Crew  Station 
Geometry  for  application  in  U.S.  A.  F.  aircraft.  The  research  team 
studied  a broad  range  of  sources  including  accident  data,  unsatisfactory 
reports,  and  aircraft  inspections.  They  concluded  that  the  most  fruitful 
efforts  were  associated  with  aircrew  generated  data. 

It  is  noteworthy  that  each  of  the  preceding  reports  cited  anonymity 
as  a crucial  requirement  for  any  data  collection  format.  Shapiro  et  al 
(I960)  modified  this  when  applying  subjective  techniques  in  a missile 
system  test  context.  They  placed  the  interviewee  in  the  posture  of  being 
an  anonymous  third  person;  thus  he  was  allowed  to  report  errors  he  had 
observed  others  commit.  This  depersonalization  mollified  the  feelings 
of  respondents  that  their  "admission"  of  performance  errors  might 
somehow  be  used  against  them. 

In  spite  of  their  differences,  all  of  the  preceding  studies  illuminated 
mission  event  incidents  in  which  the  performance  of  any  system  or  sub- 
system was  such  as  to  cause  some  form  of  mission  degradation.  The 
remaining  sections  of  this  paper  will  study  a technique  which  utilizes 
design  discrepancies  for  the  illuminating  incidents.  Although  the  objec- 
tives of  the  above  works  are  somewhat  different  from  the  present  thesis, 
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V.  PROBLEM  INCIDENT  REPORTING  TECHNIQUE 


Van  Cott  and  Kincaid  (1972)  state  that  the  essence  of  a human 
engineering  test  consists  of  answering  such  questions  as  the  following: 

"1.  Can  the  operator  use  the  equipment? 

2.  If  so  how  well? 

3.  If  not  what  is  considered  the  problem  by  the  operators  who 
are  part  of  the  system?  " 

The  basic  premise  put  forth  is  that  equipment  can  not  be  tested  and 
evaluated  independent  of  the  human  beings  who  will  work  with  it. 

The  Problem  Incident  Reporting  Technique  (Meister  and  Rabideau, 
1965)  is  aimed  specifically  at  answering  the  third  question.  It  is  closely 
related  to,  but  not  precisely  the  same  as,  the  Critical  Incident  Technique 
proposed  by  Flanagan  (1962). 

The  Critical  Incident  Technique  as  originally  postulated  consisted 
of  a procedure  for  analyzing  and  classifying  what  are,  in  essence, 
anecdotally  reported  observations.  For  the  most  part  Flanagan  used 
these  for  developing  job  requirements;  thus  he  classified  examples  of 
both  effective  and  ineffective  behavior.  However,  the  main  concern  of 
this  paper  is  to  collect  problem  incidents  (which  might  be  construed  as 
"negative"  incidents  as  opposed  to  "positive"  items)  for  the  purpose  of 
alleviating  poor  design  performance.  It  is  these  problem  incidents 
which  illuminate  operator  difficulties  and  presage  breakdowns  in  mission 
effectiveness;  thus,  they  can  be  used  to  identify  the  types  of  design 
behavior  which  should  be  carefully  evaluated  by  the  human  engineer. 
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Meister  (1965)  proposed  that  problem  incidents,  when  used  in 
conjunction  with  the  Problem  Incident  Reporting  Technique,  could  be 

* 

of  value  in  indicating  the  following: 

1.  Events  which  under  adverse  conditions  could  jeopardize 
safety  of  personnel  and/or  damage  equipment; 

2.  Human  errors  in  operating  equipment; 

3.  Operator  difficulties  in  performing  mission  tasks; 

4.  Inadequate  system  parameters  (i.  e.  job  environment, 
equipment  design)  that  affect  system  performance. 

The  remainder  of  this  paper  is  an  application  of  this  technique  to  the 

area  of  Naval  aircraft  design. 

A.  DATA  SOURCE 

Daniels  (1976)  suggested  that  the  U.  S.  Naval  Aviation  Safety 
School  at  the  Naval  Postgraduate  School,  Monterey,  California,  provided 
* a convenient  data  source  for  questions  of  aviation  design.  Classes  of 

twenty  to  forty  Naval  Aviators /Naval  Flight  Officers  convene  at  six  week 
intervals.  Each  section  is  composed  of  a professional  and  recently  pro- 
ficient group  of  fleet  aviators  that  mirrors  the  spectrum  of  Naval  air- 
craft types  and  communities.  It  thus  provides  an  inexpensive  data 
source  that  suffices  for  the  initial  problem  conceptualization. 

It  is  recognized  that  such  data  will  be  biased  somewhat  since 
safety  school  entrants  in  general  tend  to  be  more  proficient  than  the 
fleet  norm;  they  have  perhaps  mastered  a design  problem  that  could 
'hinder  the  "nugget"  aviator.  Yet  this  paper  is  concerned  with  establishing 
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the  feasibility  and  value  of  the  Problem  Incident  Technique  in  order  to 
justify  its  projection  to  the  squadron  level  (as  a fleet-wide  source  of 
data).  To  this  end  the  safety  school  provides  in  a sense  a "worst  case" 
source  of  information.  After  all  if  the  more  experienced  element  of 
the  fleet  identifies  discrepancies  they  certainly  cannot  be  excused  as 
solely  pilot  inexperience.  Such  errors  must  be  regarded  as  relevant 
and  worthy  of  further  study. 

B.  METHOD 

A method  of  exploration  was  sought  which  could  uncover  what  things 
operators  found  troublesome  with  their  equipment  and  could  also  provide 
a vehicle  for  further  hypothesis  formulation.  Thus  a method  was  needed 
to  facilitate  the  collection  and  processing  of  this  type  of  information. 

The  basic  steps  involved  are  presented  in  Figure  3 and  provide  a guide 
for  this  entire  study.  Phase  II,  the  analysis  section,  is  closely  aligned 
to  the  Critical  Incident  Technique  (Flanagan,  1962),  which  was  later 
refined  by  Meister  and  Rabideau  (1965)  into  the  Problem  Incident  Report- 
ing Technique. 

Because  of  the  cost  in  time  and  personnel,  alternatives  to  individual 
interviews  were  sought  for  data  collection.  As  noted  earlier  Vasilas 
et  al  (1953)  suggested  that  a group  orientation  method  was  highly  effec- 
tive in  facilitating  the  collection  of  reported  incidents.  The  method  uses 
an  interviewer  to  instruct  small  groups  of  airmen  in  both  the  nature  of 
the  procedure  and  the  value  of  such  a study.  The  respondents  then 
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FIGURE  3 
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describe  incidents  from  memory.  This  retains  the  advantages  of  the 
individual  interview  in  regard  to  personal  contact,  explanation,  and 
availability  of  the  interviewer  to  answer  questions. 

Daniels  (1976)  developed  a two  page  questionnaire  and  demonstrated 
its  potential  towards  obtaining  useful  results  in  a safety  school  environ- 
ment. Appendix  A contains  completed  samples  of  the  report  forms  which 
are  representative  of  the  anecdotes  elicited  during  this  study.  The 
initial  presentation  which  was  read  verbatim  to  each  group  prior  to 
report  completion  is  reprinted  in  Appendix  B.  Forty-five  minutes  was 
alloted  for  each  evolution. 

The  following  guides  to  development  were  adhered  to  in  order  to 
develop  adequate  means  of  collection  and  recording: 

1.  The  introductory  remarks  were  standardized  in  order  to 
maintain  continuity  across  safety  school  classes. 

2.  The  report  form  stresses  anonymity  in  order  to  avoid 
inadequate  responses  due  to  fear  of  reprisal,  evaluation, 
or  social  pressures. 

3.  The  initial  remarks  were  tailored  to  create  a general 
atmosphere  of  cooperation  with  the  respondents. 

4.  The  recording  technique  was  engineered  to  be  simple  and 
thus  easily  understood  by  all  participants;  it  also  required 
a minimum  of  time  and  skill. 

5.  Provision  was  made  for  eliciting  the  necessary  control 
information  - aircraft  model,  crow  position,  etc. 

6.  The  record  form  was  amenable  to  easy  handling  and 

sorting.  t 

7.  The  evaluator  was  not  searching  for  particular  problems; 
thus  there  was  no  attempt  to  direct  questions  towards  any 
specific  area. 
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C.  ANALYSIS  OF  DATA 

The  purpose  of  the  data  analysis  stage  is  to  summarize  and  des- 
cribe the  reported  incidents  in  an  efficient  manner  so  that  they  may  be  • 
applied  to  problems  of  design.  The  aim  is  to  increase  the  utility  of  the 
information  while  sacrificing  as  little  as  possible  of  its  comprehensive- 
ness, specificity,  and  validity. 

The  raw  material  for  the  analysis  consisted  of  a descriptive  report 
for  each  incident  elicited.  It  should  be  recognized  that  each  report  is  a 
unique  anecdote  describing  a unique  event.  Thus  each  incident  reported 
was  transcribed,  along  with  all  control  data,  onto  three  by  five  cards 
and  considered  as  one  data  point. 

The  procedures  used  at  this  stage  were  proposed  by  Flanagan  (1962); 
the  general  steps  involved  in  the  process  are  reproduced  in  Figure  4. 
There  are  three  primary  areas  that  are  of  critical  importance  which 
will  be  discussed  in  detail.  However  it  is  worthwhile  to  keep  in  mind 
some  observations  taken  from  a study  of  accidents  by  Thorndike  (1951): 

"Note  that  the  issue  with  respect  to  a particular  way  of 
analyzing  accidents  is  not  whether  it  is  right  or  wrong. 

Any  way  of  analyzing  and  any  set  of  categories  is  'right' 
as  long  as  the  categories  are  non-overlapping  and  are 
collectively  inclusive.  The  fundamental  issue  is  whether 
the  system  of  analysis  is  the  most  useful  one,  that  is, 
whether  it  can  best  serve  as  the  basis  for  analyses  which 
will  answer  practical  questions  and  suggest  courses  of  action 
which  are  likely  to  change  things  for  the  better.  " 

a.  Frame  of  reference 

Any  system  orientation  will  be  based  upon  the  judgment  of  the 
individuals  performing  the  analysis;  but,  in  general,  the  primary 
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Figure  4. 


GUIDE  FOR  ANALYSIS  OF  REPORTED  DESIGN  DISCREPANCIES 


1.  Select  general  frame  of  reference. 

2.  Sort  a sample  of  incidents  into  a few  piles  in  accordance  with  the 
frame  of  reference  selected. 

3.  Formulate  tentative  headings  for  major  areas. 

4.  Sort  additional  incidents  into  these  major  areas;  set  up  new 
sub-categories  as  they  appear  necessary.  During  this  process 
time  will  be  saved  if  all  incidents  which  are  so  similar  that 
they  will  remain  together  regardless  of  changes  in  category 
definitions  are  clipped  together  and  treated  as  one  unit. 

5.  After  a substantial  portion  of  the  incidents  have  been  classified, 
prepare  tentative  definitions  for  major  headings  and  generalized 
statements  for  each  of  the  main  categories  of  requirements. 

6.  Make  a tentative  selection  of  the  level  of  specificity-generality 
to  be  used  in  reporting  the  definition. 

7.  Redefine  major  areas  and  categories  as  necessary  while  incidents 
are  being  classified. 

8.  After  all  incidents  have  been  classified,  review  definitions  and 
revise  where  necessary. 

9.  Record  the  classification  of  each  incident  on  the  back  of  the  card. 

10.  Have  an  independent  check  made  on  the  classification  of  all 
incidents . 
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consideration  should  be  the  uses  to  be  made  of  the  data.  It  was  deter- 
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mined  that  the  primary  thrust  of  this  study  was  to  be  an  exploration 
into  the  problems  exhibited  across  aircraft  types  and  communities. 

The  purpose  is  to  demonstrate  that  common  problems  exist  and  that 
they  can  be  analyzed.  It  was  desired  to  determine  a design  deficiency 
stratification  that  would  encompass  each  and  every  data  point  available; 
hypotheses  could  then  be  generated  and  provide  inspiration  for  recom- 
mending further  work. 

b.  Category  formulation 

The  general  process  by  which  the  data  was  evaluated  and 
stratified  involves  category  formulation.  This  portion  of  the  procedure 
is  more  subjective  than  objective,  but,  in  general,  the  process  involves 
grouping  similar  incidents  under  major  headings,  then  into  categories, 
deriving  subgroups,  and  finally  formulating  descriptive  statements 
where  appropriate. 

The  initial  step  dealt  with  the  derivation  of  a trial  classifica- 
tion system.  Reports  published  on  studies  similar  to  the  current  effort 
(see  history)  plus  the  author's  experience  as  a fleet  aviator,  provided 
the  initial  insight,  experience  and  judgment  for  filing  a small  sample  of 
incidents  into  areas  consistent  with  the  frame  of  reference.  These  data 
aggregations  then  became  the  initial  major  headings  and  were  loosely 
defined.  Additional  incidents  were  then  classified  under  major  headings 
into  subheadings  called  categories.  This  step  was  iterative  with  regard 
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to  the  trial  and  revision  of  these  categories;  in  fact,  the  classification 
system  as  a whole  was  subjected  to  a constant  evolution  throughout  the 
entire  process.  The  goal  sought  was  a workable  classification  system 
which  encompassed  all  incidents  and  had  a non-overlapping  and  distinctly 
defined  format. 

A need  for  subgroups  within  the  categories  soon  became 
apparent.  These  were  developed  on  an  entirely  inductive  basis;  they 
consisted  of  groups  of  incidents  describing  very  nearly  the  same  design 
discrepancies. 

Each  incident  was  carefully  read  and  re-read  before  assigning 
it  to  the  system.  If  an  incident  could  not  be  readily  assigned,  a revision 
in  the  stratification  scheme  was  made  by  broadening  existing  headings, 
categories,  and/or  subgroups,  or  by  adding  new  ones  as  appropriate, 
c.  Level  of  specificity/ generality 

This  area  concerned  the  specificity  achieved  in  particular 
incidents  versus  the  simplicity  of  a relatively  smaller  but  more  useful 
number  of  headings.  Several  considerations  were  adhered  to  at  this 
level: 

1.  A clear  cut  and  logical  organization  was  sought  which 
could  be  presented  in  standard  outline  form. 

2.  Titles  were  sought  for  all  headings,  categories,  and 
subgroups  which  could  convey  meaning  in  themselves. 
Nonetheless,  definitions  and  descriptive  statements 
were  still  provided. 
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3.  The  stratification  scheme  chosen  was  that  deemed 
maximally  useful  in  highlighting  design  problem 
areas,  yet  which  was  still  significant  at  each  level 
in  terms  of  frequency  of  occurrence. 

4.  A descriptive  statement  was  written  for  each  group 
of  duplicate  incidents  or  for  each  single  incident 
representing  a distinctly  different  design  discrepancy. 

5.  Descriptive  statements  were  written  in  terms  suf- 
ficiently'general  to  cover  basic  similarities  yet 
specific  enough  to  delineate  specific  problem  areas. 

After  the  initial  system  evolution  it  became  obvious  that  the 
number  of  categories  involved  resulted  in  an  unwieldy  system.  A 
reduction  was  made  by  combining  similar  subgroups  and  writing  new 
descriptive  statements.  The  essence  of  the  subgroups  was  preserved 
but  some  specific  detail  (common  to  some  but  not  all  of  the  incidents ) 
was  lost. 


After  all  classification  was  completed  and  the  level  of  speci- 
ficity/generality deemed  satisfactory,  the  definitions  and  descriptive 
statements  were  closely  re-examined  in  terms  of  the  actual  incidents 
as  now  classified.  Alterations  were  made  where  required. 
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VI.  RESULTS 


Results  were  elicited  from  five  separate  safety  school  classes 
during  the  period  October,  1975,  to  February,  1976.  A total  of  137 
aviators  were  questioned  resulting  in  286  reported  design  discrepancies. 
The  procedure  yielded  an  average  of  2.  09  incidents  per  respondent. 

The  results  suggest  that  fleet  aviators  have  something  to  say  about 
design  deficiencies.  Fleet  experience  was  indicated  by  averages  of 
1816  total  flight  hours  per  respondent,  and  also  201  flight  hours /respond- 
ent in  the  aircraft  reported  on  during  the  past  six  months. 

The  sample  as  a whole  contained  data  points  covering  twenty-six 
different  Navy  aircraft,  an  indication  of  the  breadth  of  the  information 
across  communities.  Table  1 presents  the  relevant  control  data  in 
tabular  form  broken  down  by  individual  aircraft  and  respondent  status 
(Pilot,  NFO).  The  table  reveals  some  notable  influences  which  occur 
when  using  safety  school  data  as  a microcosm  for  the  fleet.  Pilots 
significantly  outnumber  NFO's  by  a 8 to  1 margin,  a significantly  greater 
ratio  than  might  be  expected.  Only  the  F-4  community  seems  to  have 
adequate  NFO  representation.  It  was  also  noted  that  4907o  of  the  data 
was  received  from  the  P-3,  F-4,  A-6,  and  A-4  communities.  In  fact, 

72.  8%  came  from  the  fighter,  attack,  and  ASW  communities.  However, 
these  are  also  clearly  the  main  components  of  the  present  day  force 
structure. 
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Table  II  presents  the  general  headings  and  categories  developed 
through  application  of  the  Problem  Incident  Reporting  Technique. 
Statistics  included  are  total  discrepancies  per  heading,  total  per 
category,  and  frequency  of  report  per  heading.  The  complete  analysis 
is  presented  in  Table  III.  It  consists  of  a systematically  collected  set 
of  31  subgroups,  10  categories,  and  5 headings  elucidating  those  design 
problems  viewed  as  significant  by  fleet  operators.  Each  subgroup  is 
quantified  by  number  of  discrepancies  and  frequency  of  report.  Descrip- 
tive statements  and  definitions  are  provided  where  appropriate. 

In  terms  of  frequency  or  relative  proportions,  the  grouping  under 
the  second  general  heading.  Aircraft  Cockpit  Layout  Errors,  seems 
most  important.  One  hundred  and  forty- seven  documented  incidents, 
representing  51.  4%  of  all  reported  design  discrepancies,  fall  under 
this  heading.  An  examination  of  the  subgroups  under  the  three  categories 
which  evolved  (control/display  location,  control/display  geometry,  and 
workspace  area)  reveals  problem  areas  which  might  not  be  anticipated 
by  an  engineer  lacking  aviation  experience  and  working  with  a stationary 
mockup.  Twenty-five  location  errors  which  prohibit  an  effective  visual 
instrument  scan  were  reported.  Eighteen  discrepancies  revealed  that 
certain  controls  or  displays  were  difficult  for  aviators  to  use  not  because 
of  design,  but  because  they  were  hidden  behind  other  aircraft  fixtures. 
There  were  28  reports  that  aviators  could  not  reach  or  activate  a con- 
trol when  properly  seated  in  the  aircraft.  In  addition  53  other  discrep- 
ancies were  reported  in  the  workspace  area,  an  aviator's  "office"  - 
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TABLE  II 


BREAKDOWN  OF  INCIDENTS  BY 
MAJOR  HEADING  AND  CATEGORY 


CATEGORY  HEADING 
INCIDENTS  TOTAL 


I.  EQUIPMENT  DESIGN 


A. 

Control 

39 

B. 

Display 

30 

II. 

AIRCRAFT  COCKPIT  LAYOUT  ERRORS 

A. 

Control  and  Display  Location 

60 

B. 

Control  and  Display  Geometry 

34 

C. 

Workspace  Area 

53 

III. 

VISION 

A. 

Internal 

15 

B. 

External 

8 

IV. 

ENVIRONMENT 

A. 

Environmental  Control 

13 

B. 

Life  Support 

14 

V. 

SAFETY 

A. 

Inadequate  Safety  Factors 

20 

TOTALS 

286 

69  24.  13 


147  51.4 


23  8.04 


27  9.44 


20  6.99 


286  100 
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TABLE  III 

ANALYSIS  OF  286  OPERATOR  GENERATED 
NAVAL  AIRCRAFT  DESIGN  DISCREPANCIES 


EQUIPMENT  DESIGN 


NO. 

OF 

INCIDENTS 


Control 

1.  Distinction  Errors: 

Errors  resulting  from  inability  to 
distinguish  among  switches  in  close 
proximity  and/or  confusion  among 
switches  similar  in  size  and  shape. 

2.  Adjustment  Errors: 

Operating  control  requires  excessive 
visual  attention  because  of  "lack  of 
feel"  or  because  of  excessive  physical 
effort  in  actuating. 

3.  Excessive  Positive  Actuation: 

Control  requires  a significant  time 
period  of  actuation  for  operation. 

4.  Reversal  Errors: 

Control  must  be  moved  in  a direction 
opposite  to  that  necessary  to  produce 
desired  result. 

5.  Unintentional  Activation: 

Control  positioned/designed  such 
that  danger  exists  of  inadvertently 
operating  it  without  an  awareness 
of  doing  so. 

Display 

1.  Parallex: 

Difficulty  in  reading  instrument  as 
required  because  of  angle  from 
which  it  is  viewed. 

2.  Legibility: 

Errors  which  result  from  difficulty 
in  seeing  the  desired  information 
distinctly  enough  to  interpret  indica- 
tion properly. 
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TABLE  III  - continued 

NO. 

OF 

INCIDENTS  % 


3.  Interpretation:  4 

Instrument  indication  subject  to 
misinterpretation  with  the  result  that 
subsequent  actions  may  aggravate  vice 
correct  undesirable  condition 

4.  Unreliability  From  Poor  Construction:  9 

Display  does  not  attract  attention  and/ 

or  not  utilized  due  to  poor  design. 

II.  AIRCRAFT  COCKPIT  LAYOUT  ERRORS 

A.  Control/Display  Location 

1.  Vertigo  Inducing  Location:  17 

Instrument  requires  visual  check 

causing  a distraction  which,  under 
certain  circumstances,  may  result 
in  a contradiction  between  the  infor- 
mation received  by  a pilot  through  his 
vestibular  (or  labyrinthine)  sense  and 
through  his  visual  sense. 

2.  Location  Prohibits  Effective  Scan:  25 

Control  or  display  placed  such  that 

an  effective  visual  instrument  scan 
must  be  broken  to  activate/check. 

3.  Controls,  Displays  Hidden  Behind  Other  18 
Aircraft  Fixtures 

B.  Control,  Display  Geometry: 

Problems  associated  with  dimensional 

arrangement  of  crew  station. 

1.  Inadequate  Placement,  Displays:  6 

Dimensional  layout  of  cockpit  makes 
display  difficult  to  see. 

2.  Inadequate  Placement,  Controls:  28 

Dimensional  layout  of  cockpit  makes 
control  difficult  to  reach/activate  when 
pilot  properly  seated  and  harnessed. 


1.  4 


3.2 


6 


8.  8 


6.  3 


2.  1 

9.  8 


36 


1 

I 


! 

) 


C. 


TABLE  III  - continued 

NO. 

OF 

INCIDENTS 

Workspace  Area 

1.  Operation  and  Maintenance  Consoles. 

a.  Insufficient  Instrumentation:  17 

Operator  does  not  have  neces- 
sary instrumentation  for  proper 
performance  of  his  duties. 

b.  Lack  of  Standardization:  5 

Cockpit  instrumentation  not 
standardized  across  variations 

of  the  same  aircraft  type. 

2.  Inadequate  Storage:  3 

More  stowage  space  required  for 
publications  and  emergency  equipment- 
items  which  could  become  missiles  in  a 
ditching /heavy  turbulence  situation. 

3.  Lack  of  Well  Lit,  Well  Situated  11 

Approach  Plate  Holder. 

4.  Ingress / Egress : 13 

Problems  associated  with  the  crew- 
man's ability  to  exit  or  enter  the  crew 
station  both  normally  and  during  an 
emergency. 

5.  Comfort/Mobility:  4 

Problems  associated  with  crewman's 
comfort  and  his  ability  to  move  around 

in  the  crew  station  in  order  to  perform 
his  duties. 


% 

6 

1. 75 
1.0 

3.9 
4.  6 

1.4 


III.  VISION 

A.  Internal  Vision:  15  5.3 

Problems  which  interfere  with  a crewman's 

ability  to  acquire  internal  cues  (i.  e.  , light- 
ing too  dim,  too  difficult  to  see  instrument 
indication  in  sunlight). 

B.  External  Vision:  8 2.8 

Problems  which  interfere  with  crewman's 

ability  to  acquire  external  cues  (i.  e.  , night 
display  lighting  reflects  off  canopy). 
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TABLE  III  - continued 

NO. 

OF 

INCIDENTS 

% 

ENVIRONMENT 

A. 

Environmental  Control. 

Problems  associated  with  environmental 
factors  which  have  a direct  behavior  on 
operator  performance. 

1.  Temperature/Ventilation 

5 

1. 75 

2.  Noise/ Vibration 

8 

2.  8 

B. 

Life  Support: 

Problems  associated  with  an  aviator's 

personal  equipment,  survival  equipment. 

etc. 

1.  Oxygen/Smoke  mask 

5 

1. 75 

2.  Protective  clothing 

3 

1.  0 

3.  Restraint  gear 

5 

1. 75 

4.  Relief  tube 

1 

. 035 

V.  SAFETY 

A.  Inadequate  Safety  Factors: 

Operator  identified  discrepancies  which 
could  potentially  evolve  into  conditions 


not 

conducive  to  good  safety  practice. 

1. 

Air  canopy  problems 

7 

2.5 

2. 

Training/NATOPS  procedures,  lack  of 

3 

1.  1 

3. 

Ejection  seat/seat  pan 

4 

1.  4 

4. 

Inadequate/insufficient  safety  features 
(i.  e.  , equipment  hard  to  reach  in  an 
emergency,  lack  of  weather  radar, 
lack  of  safety  checks). 

6 

2.  1 
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all  problems  that  could  have  been  avoided  by  more  knowledgeable 
direction  in  the  design  phase.  Clearly,  design  engineers  have  problems 
in  conceptualizing  the  plight  of  an  aviator  in  an  actual  operational 
environment. 

Judging  from  frequencies  the  design  of  the  control  or  display  itself 
is  next  in  importance.  Sixty-nine  such  incidents  were  analyzed  in 
Table  III.  Particularly  noteworthy  was  the  similarity  between  sub- 
groups which  evolved  from  this  report  and  those  which  Fitts  and  Jones 
(1947a,  1947b)  constructed  from  pilot  error  analyses  of  aircraft  con- 
trol operation  and  instrument  interpretation.  The  twenty-nine  year  time 
span  between  the  reports  indicates  the  necessity  for  the  transmittal  of 
this  type  of  information  to  designers.  It  also  suggests  the  absence  of 
corporate  memory  in  Naval  Aviation. 

The  discrepancies  defined  under  the  remaining  three  major  headings 
comprised  only  24.  47%  of  the  total  incidents.  Yet  it  must  be  remembered 
that  only  one  incident,  if  well  documented,  is  of  significant  importance. 

Any  factors  which  interrupt  an  aviator's  visual  perspective,  either 
internal  or  external  to  his  craft,  can  have  a degrading  effect  on  total 
system  performance.  There  were  15  reported  cases  of  excessively 
dim  lighting  at  night,  instruments  that  were  unreadable  in  sunlight,  and 
other  problems  which  interfered  with  an  aviator's  ability  to  acquire 
internal  cues.  Nine  incidents  (such  as  night  canopy  relfection  from 
display  lighting)  resulted  from  external  vision  interrupts.  Here  again 
are  disc repancies  that  would  be  difficult  to  forecast  from  a mockup  or 


simulator. 
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Also  difficult  to  predict,  but  within  the  realm  of  human  factors,  are 
the  environmental  control  and  life  support  design  problems  elicited. 
Problems  involving  temperature,  ventilation,  noise,  vibration,  and  life 
support  equipment  all  affect  a crewman's  perception  of  his  environment 
and  influence  his  performance. 

The  final  heading  was  a loosely  defined  grouping  termed  Safety. 
These  were,  in  fact,  discrepancies  which  operators  felt  could  result  in 
conditions  not  conducive  to  good  safety  practices.  Some,  such  as  lack 
of  training/NATOPS  procedures,  might  be  viewed  as  foreign  to  the 
design  process.  Yet  it  is  at  this  early  stage  where  initial  iterations 
for  such  procedures  might  be  formulated. 

Table  IV  presents  a breakdown  of  the  classification  system  versus 
the  four  aircraft  with  the  largest  sample  sizes  - P-3,  A-4,  F-4,  A-6  - 
measured  in  terms  of  frequency  of  responses  per  given  aircraft.  The 
table  gives  an  idea  of  the  dispersion  of  individual  aircraft  responses 
among  the  various  headings  and  categories.  These  models  have  all 
undergone  various  modifications;  the  A-4  and  F-4  are  in  the  twilight  of 
their  careers.  It  is  clear  that  the  classification  scheme  represents 
design  discrepancies  which  affect  all  individual  aircraft  and  yet  which 
have  gone  uncorrected  by  design  engineers  over  a substantial  period  of 
aircraft  evolution.  If  such  problems  have  not  been  pinpointed  in  the  past, 
there  is  little  reason  to  expect  them  to  be  accounted  for  in  new  aircraft. 
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TABLE  IV 


DISCREPANCY  ANALYSIS  VERSUS  FOUR  AIRCRAFT  TYPES 


% 

P-3 

% 

A-4 

% 

F-4 

% 

A-6 

EQUIPMENT  DESIGN 

A.  Control 

2.5 

17.  6 

11.  4 

6.  2 

B.  Display 

15 

3 

2.  9 

18.  8 

A/C  CONTROL  LAYOUT  ERRORS 

A.  Control/Display  Location 

10 

26. 4 

17.  1 

25 

B.  Control/ Display  Geometry 

5 

20.  6 

17.  1 

15.  6 

C.  Workspace  Area 

40 

14.  6 

14.  3 

18.  8 

VISION 

A.  Internal 

5 

3 

2.  9 

0 

B.  External 

0 

3 

2.  9 

12. 5 

ENVIRONMENT 

A.  Environmental  Control 

7.  5 

3 

8.  6 

0 

B.  Life  Support 

12.  5 

5.  8 

5.  7 

3.  1 

SAFETY 

A.  Inadequate  Safety  Factors 

2.5 

3 

17.  1 

0 

i 


It  can  also  now  be  said  that  two  types  of  requirements  arise  when 
discussing  design  problems: 

1.  Those  inspired  by  specific  aircraft  deficiencies. 

2.  Those  which  are  based  upon  a need  by  all  aircraft. 

The  current  effort  presents  a stratification  that  can  serve  as  the  founda- 
tion for  study  of  either  requirement  as  the  descriptive  statements  written 
for  this  analysis  are  generally  applicable.  A more  specific  and  detailed 
analysis  tailored  towards  either  of  the  above  directions  would  lead  to 
more  critical  discrepancy  areas  for  which  specific  remedial  procedures 
might  be  devised.  This  relates  directly  to  the  question  of,  "How  many 
incidents  should  be  collected?"  Flanagan  (1962)  suggests  a criteria 
that  would  lead  to  a much  larger  sample  than  the  present  one:  namely, 
adequate  sampling  is  achieved  when  the  addition  of  100  incidents  results 
in  only  two  or  three  new  discrepancies.  This  type  of  very  detailed  work 
could  spell  out  specific  trouble  spots  upon  which  attention  could  be 
focused  for  the  purpose  of  understanding  and  diagnosis. 
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VII.  CONCLUSIONS 


Although  the  level  of  specificity/generality  did  not  allow  specific 
identification  of  error  causing  equipment,  the  results  do  provide  a view 
of  the  problems  which  hinder  operator  performance.  The  derived  design 
deficiency  structure  demonstrated  that  an  identifiable  body  of  common 
cause  factors  exists  across  a large  number  of  different  Naval  aircraft. 
The  constant  nature  of  the  problem  suggests  that  the  causes  are  rooted 
in  the  aviation  system  itself  and  that  research  should  be  directed  towards 
conceptual  deficiencies  in  the  design  of  air  weapons  systems  in  general. 

Previous  discussion  established  a link  between  equipment  design  as 

a system  element  and  the  operator  in  terms  of  an  error  causing  overload 

/ 

conditions.  Each  of  the  design  problems  identified  in  this  paper  can 
induce  such  a state  providing  it  occurs  at  a critical  time  during  the 
mission.  It  is  not  the  deficiency  area  per  se  that  is  important,  but 
rather  the  interactive  characteristics  an  area  has  with  the  system  as  a 
whole.  As  a result  equipment  may  seem  perfectly  adequate  to  the  human 
factors  specialist  when  evaluated  alone,  while  actually  being  inadequate 
when  interacting  with  other  system  elements  (including  the  operator)  in 
the  complex  operational  environment. 

Research  needs  to  consider  not  only  specific  problems  of  equipment 
design,  but  to  study  a system  in  terms  of  the  interaction  of  its  elements. 
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System  breakdowns,  as  well  as  individual  equipment  inadequacies, 
directly  affect  mission  performance.  Thus  the  Navy  needs  a system 
development  program  which  includes  consideration  of  total  system 
operation  (rather  than  merely  operability)  from  the  earliest  of  design 
stages.  A start  has  been  made  in  this  direction  by  the  approach  presented 
in  this  study;  deficiencies  as  deduced  by  the  presented  method  reveal  a 
unique  view  of  problems  as  experienced  in  a systems  context. 

With  regard  to  the  presented  collection,  classification, and  analysis 
of  the  deficiency  data,  the  author  concluded  that: 

1.  Fleet  aviators  do  feel  that  man-machine  interface  problems 
exist  which  hamper  their  mission  performance. 

2.  Aviators  are  able  and  willing  to  articulate  and  document 
these  problems. 

3.  The  Problem  Incident  Reporting  Technique  contributes  to 
an  effective  method  for  systematically  collecting,  analyzing, 
classifying,  and  describing  the  reported  problem  incidents. 

4.  The  continuation  of  the  same  discrepancy  problem  areas 
across  generations  of  Naval  aircraft  and  their  continued 
inclusion  in  design  suggests  that  operational  problems  do 
exist  which  cannot  be  anticipated  by  the  design  engineer 
using  current  techniques. 

1 

5.  The  presented  method  can  identify  these  operational  problem 
areas  which  have  been  historically  ignored. 
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7. 


The  utilization  of  such  information  in  the  design  stage  can 
result  in: 

a.  improved  performance  in  weapon  systems  cockpit  and 
the  man-machine  interface  design, 

b.  improved  manpower  utilization  and  performance, 

c.  fewer  aircraft  mishaps  and  an  alleviation  of  mission 
degradation  from  equipment  misuse, 

d.  improved  user  acceptance, 

e.  reduced  training  cost  as  aviators  no  longer  must  learn 

| 

methods  devised  to  circumvent  design  problems. 

Thus,  a system  has  been  devised  for  the  total  analysis  of 
reported  design  discrepancies  which  yields: 

a.  a new  source  of  specific  and  detailed  data  which  provides 
considerably  more  information  on  important  design  prob- 
lems than  those  currently  available. 

b.  a system  for  the  classification  and  analysis  of  the  data 
in  such  a way  as  to  facilitate  remedial  action. 


45 


1 


VIII.  RECOMMENDATIONS 


This  project  represents  the  second  iteration  in  developing  a pro- 
gram for  obtaining  and  utilizing  design  discrepancy  data.  The  study 
used  a questionnaire  developed  by  Daniels  (1976);  however,  the  expe- 
rience accumulated  in  the  course  of  this  work  indicates  that  valuable 
information  was  left  undetected.  Chapanis  (1959)  suggests  that  a pro- 
ject of  this  nature  should  elicit  data  that  suggests  cures.  And,  in  fact, 
many  aviators  offered  plausible  solutions  to  the  design  problems  which 
they  described.  In  addition  it  would  have  been  advisable  to  ask  more 
specific  questions  in  order  to  determine  the  phase  of  the  mission  which 
was  affected,  what  the  aviator  did,  and  in  what  ways  the  problem  de- 
graded the  flight.  In  this  way  an  idea  of  the  criticality  in  terms  of 
mission'performance  could  be  devised.  It  is  suggested  that  this  be 
dealt  with  in  a series  of  follow  up  projects;  however,  a solid  starting 
point  would  be  a record  form  developed  by  the  American  Institute  for 
Research  and  used  by  Vasilas  et  al  (1953^  in  a study  of  near  accidents 
(see  Appendix  C). 

Eventually  the  study  should  be  expanded  to  collecting  design  dis- 
crepancy data  from  all  crew  members  on  a fleet-wide  basis.  Practical 
procedures  and  channels  must  be  devised  for  routing  reports  to  squad- 
rons, for  application  within  squadrons,  and  for  routing  back  to  the 


collection  agencies . In  addition  channels  for  effecting  the  corrective 
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actions  dictated  by  the  analysis  must  be  established.  The  facilities  of 
the  Naval  Postgraduate  School,  the  Naval  Safety  School,  and  the  Naval 
Air  Development  Center  should  be  explored  with  respect  to  establishing 
such  procedures. 

It  must  be  recognized  that  the  data  used  in  such  a study  as  this  is 
subjective.  If  is  recommended  that  the  design  discrepancies  which 
result  from  such  data  be  substantiated  with  other  observable  data  and 
documented  reports.  Aircraft  inspections  could  be  conducted  and  safety 
data  screened  for  such  validation  purposes.  The  process  when  bolstered 
by  additional  information  can  only  gain  in  credibility. 

Finally  it  is  recommended  that  the  resultant  findings  be  forwarded 
to  the  human  engineer  with  the  recommendation  that  the  solutions  be 
included  in  the  design  specification.  Meister  and  Farr  (1966)  in  a study 
of  the  utilization  of  human  factors  by  designers  suggest: 

"The  primary  means  of  ensuring  the  inclusion  of  Human  Factors 
in  design  is  through  the  design  specification  ....  We  recom- 
mend that  when  consideration  of  Human  Factors  is  desired,  the 
customer  insert  strongly  worded  statements  in  the  design 
specification " 

If  an  effective  communications  link  is  established  from  the  squadron 
through  the  collecting  agency  to  the  contractor,  perhaps  the  design 
problems  that  have  plagued  Naval  aviators  through  generations  can  be 
mitigated. 
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APPENDIX  A 


COCKPIT  HUMAN  FACTORS  DEFICIENCY  QUESTIONNAIRE 


SECTION  I - RESPONDENT  DATA 
# 85  Date  19  Jan. 

NFO  (check  one) 

Total  Flight  time  2150  . A., 

6 A-4M 

Aircraft  flown  most  in  last  6 months.  Type  (ECP  1120) 


Hours  200 


Do  you  think  the  data  we  are  trying  to  get  is  available  elsewhere?  Yes  No 
If  yes,  where?  


i 

! 


' 

i>'  • 

3 

' 

t 


SECTION  II  - IDENTIFICATION  OR 

DESCRIPTION  OF  DEFICIENCY 

ITEM  1 

ITEM  2 ITEM  3 

ITEM  4 

Type(s)  of  aircraft  involved 

A-4M 

A-4M  A-4M 

A-4M 

Name  of  deficient  item 

ARC-159 

(placement 

nf) 

nose\yheel  fuel 
steering  trans 

button  Tight- 

AOA 

ind. 

Component  ID  (if  known) 

(dual 

function) 

Manufacturer  (if  known) 

Douglas 

Douglas 

* 

Was  this  item/ situation  standard? 

Yes  No 

i.  e.  Did  it  occur  on  all  aircraft  in  type?  yes 

yes  yes 

yes 

If  NO,  describe  below 

SECTION  III  - BRIEFLY  DESCRIBE  THE  SITUATION  AND  ITS  POTENTIAL 

EFFECTS 

Item  1 

"In  the  new  A-4M  with  ECP  1120  (HUD),  the  cockpit  lay-out  has  improved 
but,  there  are  still  some  environmental  problems.  The  ARC  159  a new  UHF 
radio  is  the  best  I have  ever  had  the  opportunity  to  use.  But,  it  is  still 
located  down  on  the  console;  it  was  moved  from  the  right  console  to  the  left 
which  was  an  improvement  - facilitating  keeping  right  hand  on  stick.  How- 
ever, it  is  just  about  abeam  one's  left  upper  thigh.  When  making  a channel 
change  especially  night,  IFR,  formation  it  is  necessary  to  look  down  and  to 
the  left  to  read  the  new  frequency.  This  is  dangerous  and  impractical. 

Mount  the  radio  on  the  left  upper  instrument  panel  or  have  a readout  from 
the  radio  displayed  in  that  position.  The  OV-10  is  a good  example  of  radio 
placement.  It  makes  it  much  more  comfortable  and  less  vertigo  inducing 
to  switch  frequencies  while  maintaining  good  look-out  doctrine.  " 
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APPENDIX  A - continued 


Item  2. 

"The  nose  wheel  steering  button  has  a dual  function  in  A/C  with  the 
(HUD).  During  weapons  delivery  if  the  master  armament  switch  is 
cycled  on-off,  as  in  coming  off  a run,  the  pipper  and  bomb  actuation 
line  disappears.  In  order  to  regain  this  presentation  the  pilot  must 
depress  the  nosewheel  steering  button.  No  one  knows  why  this  feature 
was  even  incorporated  to  begin  with.  The  presentation  does  not  impair 
one's  view  and  as  a result  of  depressing  the  nosewheel  steering  button 
many  pilots  have  already  inadvertently  depressed  the  bomb  pickle  button 
dropping  ordnance  anywhere  and  everywhere.  I suggest  removing  this 
feature  of  the  nosewheel  steering  (dual  purpose)  and  merely  leaving  the 
pipper  and  bomb  actuation  line  always  displayed.  All  pilots  in  my  present 
squadron  concur  - it's  a ridiculous  feature.  " 


Item  3. 

"There  is  no  longer  a fuel  transfer  light  in  the  new  A-4M  displayed  on 
the  enunciator  panel.  The  NATOPS  makes  reference  to  it  still  being 
there  but  it  is  not.  A change  has  already  been  submitted  to  the  NATOPS 
manual.  However,  the  pilots  would  like  to  have  the  light  back.  There 
is  presently  no  way  to  discriminate  between  a stuck  float  valve  and  a 
fuel  transfer  failure.  They  are  both  indicated  by  a drop  in  fuel  quant, 
to  approx.  1100  lbs.  followed  by  a low  fuel  light.  " 


Item  4. 

"The  AOA  indicator  is  in  a poor  position  for  the  pilot  to  see  when  seat 
is  in  upper  position  for  landing.  Raise  the  whole  indicator  for  better 
landing  A/S  monitoring.  " 
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APPENDIX  A _ continued 

COCKPIT  HUMAN  FACTORS  DEFICIENCY  QUESTIONNAIRE 
SECTION  I - RESPONDENT  DATA 
Date  10  Nov  75 
NFO  (check  one) 

Total  Flight  time  4500 

Aircraft  flown  most  in  last  6 months.  Type  P-3C  Hours  30 

Do  you  think  the  data  we  are  trying  to  get  is  available  elsewhere? 
Yes  No  . If  yes,  where? 
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SECTION  II  - IDENTIFICATION  OR  DESCRIPTION 

OF  DEFICIENCY  ITEM  1 ITEM  2 ITEM  3 


Type(s)  of  aircraft  involved 


Name  of  deficient  item 


P-3C  P-3C  P-3C 

Approach  iiight  vertical 

plate  director  speed 

holder  indicator  indicator 


Component  ID  (if  known) 


Manufacturer  (if  known) 


Was  this  item/ situation  standard? 

Yes  No  yes  y e s yes 

i.  e.  Did  it  occur  on  all  aircraft  in  type? 

If  NO,  describe  below 


SECTION  III  - BRIEFLY  DESCRIBE  THE  SITUATION  AND  ITS  POTENTIAL 
EFFECTS 
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APPENDIX  A - continued 


Item  1 

"In  the  P3C  aircraft  there  is  no  holder  provided  for  the  approach 
plate/ departure  plate  book.  Many  'made  do'  adaptations  have  been 
designed  and  used  by  various  individuals.  These  jury  rigs  usually 
involve  clipping  a sheet  aluminum  plate  somewhere  under  the  glare- 
shield  or  at  the  edge.  Most  of  these  locations  cause  certain  instrument/ 
indicators  to  be  hidden  from  the  pilots  scan.  Also  none  of  these  locally 
manufactured  solutions  are  lighted  causing  night  approaches  to  be  less 
safe  by  the  pilot/co-pilots  moving  the  approach  plate  to  a light  source.  " 


Item  2 

"The  Flight  Director  Indicator  (FDI)  in  the  P3C  provides  the  primary 
horizon  reference  under  IFR  conditions.  The  'pipper'  is  non-adjustable 
by  the  pilot.  Depending  on  the  height  -of- eye  of  the  viewer  it  will  in- 
dicate a climb  or  descent  when  in  fact  the  aircraft  is  straight  and  level. 
It  causes  instrument  fixation  and  disorientation  during  the  instrument 
flight.  Without  an  operational  AFCS  it  is  extremely  difficult  to  fly 
straight  and  level.  " 


Item  3 

"The  pilots  VSI  is  too  small  for  its  position  on  the  instrument  panel 
It  should  either  be  the  larger  type  gauge  or  be  repositioned.  Note: 
They  might  have  taken  the  c/p  VSI  out  of  the  P3C.  I'm  not  sure  as 
I haven't  had  a chance  to  fly  in  the  rt.  seat.  )" 
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APPENDIX  B 

INSTRUCTIONS  FOR  FILLING  OUT  THE  COCKPIT  HUMAN  FACTORS 

DEFICIENCY  QUESTIONNAIRE 

There  is  presently  no  large  data  base,  from  the  pilot's  point  of 
view,  for  problems  that  the  pilot  faces  caused  by  the  cockpit  and  its 
equipment.  It  may  be  too  late  or  expensive  to  change  the  specific  items 
or  address  specific  problems  that  relate  to  known  deficiencies  today. 
However,  the  hope  is  that  a data  base  made  up  of  such  problems, 
statistically  relevant  and  properly  presented  in  the  DESIGN  stage  of 
tomorrow's  cockpits  may  save  us  from  having  these  same  or  similar 
problems  on  our  next  tour. 


Now  specifically,  think  back  on  your  last  flight  experience  and 


remember  some  cockpit  piece  of  equipment,  murphy,  or  design 
feature  that  was  a problem  (couldn't  reach,  see,  feel,  hear,  actuate, 
etc.  ) Almost  any  specific  item  that  caused  some  difficulty,  lack  of 
coordination,  hazard,  noise,  temperature,  etc.  will  be  helpful.  If 
you  have  two  or  three,  so  much  the  better.  Space  is  provided  for 
three  items. 

Everybody  got  an  item?  If  not,  keep  thinking  back  on  flights 
until  you  get  at  least  one.  OK,  carry  on! 


APPENDIX  C 


HAZARDOUS  INCIDENT  REPORT 


(1)  Base: 

Acft  type:  Date: 

check 

( ) or  complete  the  following  applicable  data: 

DAWN 

DAY 

DUSK  NIGHT  Clearance:  VFR  IFR  LOCAL 

PHASE  OF  FLIGHT: 

TAXIING  TAKEOFF  CLIMB  CRUISE  DESCENT  APPROACH 
LANDING  OTHER  TYPE  OF  MISSION: 


NARRATIVE  DESCRIPTION  OF  INCIDENT 
(2)  Give  enough  details  so  that  the  story  will  be  clear.  Use  other  side 
if  more  space  is  needed. 


A. 

Describe 

what  led  to  the  hazardous  incident 

B. 

What  did 

you  do? 

C. 

What  was 

the  immediate  result? 

D.  What  would  be  the  best  way  of  handling  the  situation  if  it  happened 
a gain? 


E.  How  could  such  a hazard  be  eliminated  or  avoided'3 * * * 7 


(3)  Check  your  crew  position 

Pilot  Co-pilot  Bombadier  Navigator  Observer 

Radar  operator  Engineer  Gunner  Other  


Developed  by 

The  American  Institute  for  Research 


1 

. A 
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